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The assessment of the reliability of Monte Carlo simulations is discussed, 
with emphasis on uncertainty quantification and the related impact on experi- 
mental results. Methods and techniques to account for epistemic uncertainties, 
i.e. for intrinsic knowledge gaps in physics modeling, are discussed with the 
support of applications to concrete experimental scenarios. Ongoing projects 
regarding the investigation of epistemic uncertainties in the Geant4 simulation 
toolkit are reported. 
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1. Introduction 

The investigation and quantification of epistemic uncertainties 1 is well es- 
tablished in the domain of deterministic simulation, but it is a relatively 
new domain of research in the context of Monte Carlo simulation. It con- 
cerns the issue of how epistemic uncertainties, i.e. uncertainties due to lack 
of knowledge, namely in modeling physics processes, affect the outcome of 
Monte Carlo simulation. Epistemic uncertainties are present in Monte Carlo 
codes, when the absence of experimental data, or inconsistencies in available 
measurements, prevent the achievement of firm conclusions regarding the 
correct values of physics parameters or the validity of physics models used 
in the simulation. Epistemic uncertainties can induce systematic effects in 
the simulation; this issue is especially important, since can negatively affect 
the accuracy and reliability of simulation results. 

Due to their intrinsic nature, related to lack of knowledge, epistemic 
uncertainties are difficult to quantify. Despite their importance in complex 
systems, there is no generally accepted method of measuring epistemic un- 
certainties and they contribute to the reliability of the whole system. A 
variety of mathematical formalisms 2 has been developed for this purpose; 
the most common methods adopted in the context of deterministic simu- 
lations are interval analysis and applications of Dempster-Shafer theory of 
evidence. 3 Nevertheless, these techniques may not always be directly appli- 
cable in identical form to the treatment of epistemic uncertainties in Monte 
Carlo simulations. 

Sensitivity analysis 4 is a tool for exploring how uncertainties influence 
the model output. This approach is adopted in two exploratory projects, 
which intend to evaluate possible methods for uncertainty quantification 
related to the Geant4 5 ' 6 simulation toolkit. Epistemic uncertainties are 
usually represented in statistical analyses as a set of discrete possible or 
plausible choices; in the exploratory analyses described here the possible 
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choices concerned the values of physical parameters or a set of alternative 
physics models. 

2. Proton depth dose simulation 

This study assesses the impact of epistemic uncertainties associated with 
various physics models and parameters relevant to Monte Carlo codes 
through the simulation of a concrete use case: the depth dose profile in 
water generated by a proton beam as in a typical therapeutical facility. For 
this purpose the geometry of a realistic hadrontherapy facility 7 publicly 
available MS (X Geant4 example was utilized. 

A sensitivity analysis has examined the response of the system to a wide 
set of modeling approaches; this method plays a conceptually similar role to 
the interval analysis method applied in deterministic simulation, where pa- 
rameters subject to epistemic uncertainties are varied within bounds. The 
environment for this analysis has been realized in the context of a Geant4- 
based application; the characteristic of Geant4 as a toolkit, encompassing 
a wide variety of physics models, allow the configuration of the simula- 
tion with a large number of different physics options in the same software 
environment. The outcome associated with the various models subject to 
investigation has been compared by means of rigorous statistical analysis 
methods to quantitatively estimate the effect of physics-related systematic 
uncertainties. 

Epistemic uncertainties are associated with parameters used by the sim- 
ulation models: proton stopping powers and the water mean ionization po- 
tential, for whose values a consensus has not yet been achieved in the scien- 
tific community. The interval analysis has highlighted a shift in the position 
of the Bragg peak related to range of variability of these parameters. 

Nuclear interactions, both eleastic and inelastic, affect the shape of the 
depth dose distribution. Epistemic uncertainties in this domain derive from 
the still incomplete validation of the hadronic models used by the simula- 
tion. No statistically significant effects on the depth dose profile have been 
identified as a result of the interval analysis; nevertheless, significant sys- 
tematic differences deriving from epistemic uncertainties are observed in 
other features of the simulation outcome, such as secondary particle pro- 
duction. Multiple scattering modeling also plays an important role in the 
evaluation of possible sources of systematic effects. 

Sensitivity analysis as applied to this simulation topic contributes to 
identify and quantify possible systematic effects in the simulation; it cannot 
infer anything about the validity of any of the physics models, for which 
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experimental data would be needed. 

The analysis of the proton depth dose simulation shows that the ap- 
pearance of systematic effects generated by epistemic uncertainties in the 
physic models depends not only on the intrinsic characteristics of the un- 
certainties, but also on the characteristics of the simulation environment. 

3. Atomic binding energies 

General purpose Monte Carlo codes use a variety of compilations of atomic 
electron binding energies, either deriving from theoretical calculations or 
from empirical evaluations of direct and indirect experimental data. Despite 
the fundamental character of these atomic parameters, there is no consen- 
sus among the various Monte Carlo systems and physics models about their 
values: the differences across the binding energies reported in the various 
compilations range from the electronvolt scale to several hundred elcctron- 
vonvolts. 

The analysis adopted two complementary approaches: on one side di- 
rect validation based on binding energies measurements, on the other side 
the evaluation of how different compilations of these parameters contribute 
to the accuracy of physics observables calculated by the simulation with 
respect to experimental data. 

Reference experimental values for direct validation of atomic binding en- 
ergies are relatively limited: the main issue for direct validation consists of 
discrepancies in experimental values due to calibration effects, for instance 
when measurements are taken in different laboratories and exploit different 
experimental techniques. Two sets of reference data concerning core shells, 
which have been subject to a process of recalibration and evaluation, have 
been assembled by Powell 8 and NIST (United States National Institute 
of Standards), 9 which encompass respectively only 65 and 81 binding en- 
ergy values. In addition, NIST reports reference ionization energies for all 
elements. 10 

Direct comparison of the binding energies in the various compilations by 
means of statistical methods has identified the compilation by Williams 11 as 
the one, among those considered in this study, exhibiting the best compat- 
ibility with Powell and NIST reference data. Regarding ionization energies, 
the compilation by Carlson 12 appears to best reproduce NIST reference val- 
ues. Characteristic X-ray energies are best reproduced by the compilation 
by Larkins. 13 

The values of atomic binding energies can significantly affect the ac- 
curacy of ionization cross section calculations, both for electron and pro- 
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ton impact ionization. Among the compilations subject to analysis, EADL 
(Evaluated Data Library) 14 contributes to deteriorate the accuracy of ion- 
ization cross sections with respect to empirical compilations. 

Characteristic K and L-shell X-ray transition energies are more accu- 
rately calculated by using binding energies compiled by Larkins. No signif- 
icant effect depending on the choice of binding energies is observed in the 
photon spectrum in Compton scattering accounting for Doppler broaden- 
ing. 

4. Conclusions 

The exploratory analysis of epistemic uncertainties in two Monte Carlo sim- 
ulation domains has highlighted their contribution to simulation accuracy 
and their capability of generating systematic effects in simulation results. 
Further investigations are in progress to identify and quantify epistemic 
uncertainties in Geant4 physics models. 

Due to the limited page allocation in these conference proceedings, the 
detailed results of the analysis cannot be reported here; they can be found 
in dedicated publications. 15,16 
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